Skip to content

[tablet, queryrules] Extend query rules to check comments#8233

Merged
ajm188 merged 2 commits intovitessio:mainfrom
falun:falun-rules-update
Jul 22, 2021
Merged

[tablet, queryrules] Extend query rules to check comments#8233
ajm188 merged 2 commits intovitessio:mainfrom
falun:falun-rules-update

Conversation

@falun
Copy link
Copy Markdown
Contributor

@falun falun commented Jun 2, 2021

⚠️ Caveat is that I pulled the PR and description from #7784 since it was accidentally merged that into upstream instead of the TS branch so it's a little non-trivial to just reopen it due to the revert (afaik). I did not do any additional retesting relying on the unit tests etc to do their thing.

Description

This PR makes three changes

  1. It introduces an optional us of fsnotify to watch for changes vs the rules file
  2. it extends rules evaluation to support comparison vs leading/trailing comments
  3. It adds support to specify leading/trailing comments as part of rule creation in rulesctl

Details: Watching the file

We use fsnotify which seems to be the generally accepted approach. There are some gymnastics necessary as watching the specific file is prone to losing track of it in the event that the file is deleted+recreated. To handle this case we watch the directory containing the file and react only when the node we receive events for matches the file we are watching.

Details: Filtering on MarginComments

This was mostly plumbing within the Rule itself + the application callstack as the tabletserver/query_executor::QueryExecutor has already baked margin comments into it during the initial phase of the Execute call.

Evaluation is similar to but not the same as the Query regexp because that is stable across query shapes (otherwise it wouldn't be the same query). For margin comments we need to defer evaluation until after the plan cache which is why Query & comments take different paths.

Test failures

The only listed failure is endtoend/tabletmanager/tablegc which seems unlikely to be related afaict.

Related Issue(s)

n/a

Checklist

  • Should this PR be backported?
  • Tests were added or are not required
  • Documentation was added or is not required.

Deployment Notes

No changes necessary for existing deployments.

Impacted Areas in Vitess

Components that this PR will affect:

  • Query Serving
  • VReplication
  • Cluster Management
  • Build/CI
  • VTAdmin

@harshit-gangal harshit-gangal added Component: Query Serving Type: Enhancement Logical improvement (somewhere between a bug and feature) labels Jun 2, 2021
Copy link
Copy Markdown
Member

@harshit-gangal harshit-gangal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

overall looks good. added some comments.

…nt merge.

Signed-off-by: falun <635538+falun@users.noreply.github.com>
@falun falun force-pushed the falun-rules-update branch from 072a8eb to 7d3dcfe Compare June 6, 2021 08:01
Signed-off-by: falun <635538+falun@users.noreply.github.com>
@falun falun force-pushed the falun-rules-update branch from 7d3dcfe to d10ce23 Compare June 6, 2021 08:14
Comment on lines 108 to +115
if *fileRulePath != "" {
qsc.RegisterQueryRuleSource(FileCustomRuleSource)
fileCustomRule.Open(qsc, *fileRulePath)

if !*fileRuleShouldWatch {
return
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: same change could be made for fileRulePath as was done for fileRuleShouldWatch

@ajm188 ajm188 merged commit f2da287 into vitessio:main Jul 22, 2021
ajm188 pushed a commit to tinyspeck/vitess that referenced this pull request Jul 22, 2021
[tablet, queryrules] Extend query rules to check comments

Signed-off-by: Andrew Mason <amason@slack-corp.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Component: Query Serving Type: Enhancement Logical improvement (somewhere between a bug and feature)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants